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Abstract 

Given a set of n elements we investigate how much of the ordering can be deter¬ 
mined by an instrument that takes k elements and returns the tf 
of them. We consider this question in both an on-line sense, in which future 
choices can depend on previous results, and off-line where all the queries must 
be chosen initally before knowing any results. 
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Our aim in this paper is to study the following question. Assume a user has 
an ordered set of n elements in which the ordering is fixed but not known (for 
example distinguishable but unmarked coins of unknown distinct weights), and 
that he wishes to determine the ordering. We denote this base set X, containing 
elements xi, X 2 ... Xn- He is given a scale that accepts as input a fc-set of ele¬ 
ments and returns a fixed subset of them according to the ordering, for example 
it might return a subset of size s that contains the tf" ,t 2 '^, ..., elements. We 
would call such a scale a ... ,ts) scale, and wish to know what one can 

determine about the ordering of the elements from repeated use of such a scale. 
We shall refer to the process of using the scale on a fc-set as querying that set. 
These have been previously studied in the case where fc = 2, in which case they 
are known as binary scales. In this paper we expand this to consider the case 
where k is greater than 2, and give algorithms to efficiently determine as much 
of the ordering as possible. We analyse the order of the number of queries for 
fixed k and large n in both the online and offline settings. 

Clearly the user cannot completely discover the ordering, as you cannot deter¬ 
mine the ordering of the first — 1 elements or the final {n — tg) elements, if 
> 1 and {n — ts) < 1- Let us call these sets S for the initial segment of 
‘small’ elements and L for the final segment of ‘large’ elements. Additionally 
note that if the scale is symmetric (in the sense that the elements that it re¬ 
turns are symmetric around the midpoint of k i.e. ti = k — ts,t 2 = k — G-i 
etc) then the ordering cannot be fully determined for the remaining elements, 
as the results of any query would be the same if the ordering was reflected. We 
therefore ignore this case, and assume assymmetric instruments in general. We 
also assume that the scale returns an unordered set, {fi,..., G}, rather than an 
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ordered one. This is because we show that even with an unordered set you can 
recover the full ordering of the elements; an ordered output would be strictly 
stronger, and so also able to do the same. 

A related question was considered by Hannasch, Kim and McLaughlin [1] in 
2010. They considered an instrument which again accepted an input of k- 
elements but returned the complete ordering of the input set. They asked, given 
such an instrument, how long it would take to determine to first t elements of an 
n-set, which they called S(n, fc, t). Although related their instrument is stronger, 
and the question asked weaker, so we consider our problem a more general ver¬ 
sion of theirs. We do not know of any prior work done on this, stronger, question. 

This paper is structured as follows. In section 1 we consider online algorithms, 
where the sets submitted to future queries can depend on past results. Begin¬ 
ning with the case s = 1, i.e. where the scales output a single element, we show 
that it is possible to determine the ordering of the elements (excluding the or¬ 
dering of S and L) in 0{n log n) time, the same order of bound as in the binary 
case. We give this constant explicitly, showing that it is an improvement over 
the binary case. We also investigate the case where s > 1, and give an explicit 
algorithm for determining the ordering in this case. 

In section 2 we consider offline algorithms, where all the queries must be specified 
in advance and then the full set of results are returned simultaneously. This 
could be applicable in a situation where queries have to be sent to a laboratory 
to run overnight. Obviously this requires more queries in general. In the case 
where s = I, i.e. where we are using a (/c, <)-scale, we outline an algorithm 
that works in queries, and show that this is the best possible order. 

This algorithm relies heavily on a recursive approach, determining the results 
of queries that have not been carried out from those that have. We also outline 
an alternative algorithm that works in a similar amount of time, but works 
directly, determining the ordering of the elements using an adjacency based 
argument. In the case s > I the recursive approach can often still be applied, 
but the calculations involved get more complicated and require more detailed 
case analysis. However the adjacency argument continues to work, giving us a 
general offline algorithm that works when the scale outputs an unordered set. 

1. Online Algorithms 

In this section we consider online algorithms, where the user is given the result of 
each query as he requests it, and on that basis selects the next set of k elements 
that he wants to query. As highlighted in the introduction, these scales cannot 
in general determine the full ordering of the element set, as the first and last 
segments of the ordering will never be returned by any query, and so no query 
can determine their order. We refer to these segments as S and L respectively, as 
mentioned in the introduction, and in each case shall highlight which elements 
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they comprise. The remaining elements we denote X', and note that there are 
at least n — (A: — 1) of them, which we call n'. 

We begin by considering a singleton-scale, which returns a singleton output. 
1.1. Singleton Output Scales 

The classical version of this question is the binary scale which accepts as input 
two elements and returns the smaller. We consider more general (fc, t) scales, 
accepting k elements and returning the smallest. To ease notation through¬ 
out this section we shall assume that t ^ k/2, i.e. it lies in the first half of 
the queried set. If t > fc/2 then the following analysis still holds, inverting the 
roles of S and L and making occasional other similarly minor adjustments to 
the calculations. 

We describe an algorithm that works in O(n'logn') queries to determine the 
ordering of the element set. This algorithm works by first determining S and 
L (in Stages 1.1 and 1.2), and then using them to iteratively determine the 
ordering by repeatedly determining the smallest element among the remaining 
unsorted elements (in Stage 1.3). Stage 1.3 shall take the longest, it is this stage 
that takes O(n'logn') time to run, while Stage 1.1 is linear in n' and Stage 1.2 
is independent of n’, taking a number of queries that is just a function of k. 
Hence the combination of the three works in O(n'logn') queries as required. 

Stage 1.1. Determine the elements that comprise S U L. 

Method: Note that these elements are precisely those that can not be returned 
by any query. Hence they can be identified by eliminating all the others. The 
user repeatedly picks a fc-set from those that he has not yet eliminated, and 
eliminates whatever is the output. Each time he does this it eliminates 1 more 
element. He continues doing so until he cannot find another fc-set, which occurs 
when there are k — 1 elements left. But note that S' U L is always contained 
within the remaining elements, and there are fc — 1 elements in S U T, so it is 
exactly the remaining elements at this point. 

Stage 1.2. Partition SUT into S and T, and if possible identify which is which. 

Method: Pick an arbitrary set from the eliminated elements, which we denote 
oi,... ,afc_i where the index respects the ordering of the elements. For each 
element in S U L query it taken together with {oi,..., a^-i}. If it was in S this 
will return at-i and if it was in L it will return at. Although the user does not 
know which is which, he can partition S U L into S and L according to which 
response he gets. He can further keep count of how often each comes up, as 
he will receive at_i [S'! times, and at \L\ times. If [S'! ^ \L\ then this further 
identifies S and L — the only time this won’t work is when [S'! = \L\. This occurs 
when t = k/2, which means that the underlying scale is symmetric. Hence by 
the end of Stage 1.2 in the case of an asymmetric instrument the user knows S 
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and L, and if the instrument is symmetric then he has identified the set {S,L} 
but does not know which is which. 

Stage 1.3. Use S to determine the order of the remaining elements. 

Method: First, let us assume that our instrument is asymmetric, and hence the 
user knows S before starting this Stage - at the end we shall address what is 
done in the symmetric case. In this stage the user shall repeatedly use S to 
determine the smallest element of a subset of A:' := k — {t — 1) elements by 
querying them taken together with S. If he wishes to find the smallest element 
of a smaller set he takes it together with S and supplements it with elements 
from L until he has k elements allowing him to query it. In both cases as S 
takes up the first t — 1 elements of the queried set, it will return the next largest 
which is the smallest of the subset he is trying to check. 

The user now take the remaining n' elements and partitions them into as many 
sets of size k' as possible, with a remainder set of at most k' elements, which 
we shall call level 1 sets. He then groups the level 1 sets into sets of size k' 
(again as far as possible, with perhaps a deficient remainder set) to get level 2 
sets, so a level 2 set consists of k' sets each of k' elements. He continues this 
process until all the n' elements are in a single set - we denote the level where 
this occurs as d, where d = log^,/ n'This effectively creates a d-dimensional grid 
of sets. We illustrate this grouping for the first two layers in Figure 1. 



Figure 1: First two layers of the grouping process 


At the hrst step he queries all the level 1 sets, establishing which is the smallest 
element of each. He then ‘checks’ each level 2 set in turn by querying the small¬ 
est element of each level 1 set inside it to find the smallest element in each level 
2 set. Continuing in this manner he can establish the smallest element in every 
level r set for all 1 < r < d, and hence the smallest element in the level d set. 
But the level d set contains everything, so he has found the smallest element 
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in the remaining n' elements. He now wants to remove this element and find 
the next smallest. Note that this only requires him to check/query the level 
1, 2,..., d sets that the previous smallest was in, as the others are unaffected. 
Repeating n' times therefore determines the order of the remaining elements. 

This completes the analysis of the asymmetric case. If the instrument is sym¬ 
metric, then recall as discussed earlier that the user would only at best be 
looking to determine the ordering or its reflection, since he would not be able 
to distinguish these with any query. If his instrument is symmetric though the 
main difference is that at the start of Stage 1.3 he wouldn’t know which set was 
S and which was L. However he can arbitrarily assume either one is S, and 
carry out Stage 1.3 under that assumption. If he was correct he will get the 
correct ordering, if incorrect he will get the reflection, and given that he cannot 
distinguish these with a symmetric instrument anyway that therefore gives him 
the best possible information he could gather about the ordering. 

It remains to show that this method only uses 0(n'log(n')) queries. Stage 1.1 
requires one query for each element in \ (S' U L), so uses n — (fc — 1) queries. 
Stage 1.2 requires one query for each element in S U L, so uses {k — 1) queries. 
Thus together Stages 1.1 and 1.2 use a total of n queries. 

Stage 1.3 takes longer, this is where the extra log factor comes in. The first run 
through of the levels in which the user finds the smallest value in each set takes 
this many queries 
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< dn' 


Having completed these queries, and thus found the smallest element in the 
set X \ (S' U L), the user then needs to carry out d additional queries for each 
remaining element. Hence he requires at most d{n' — 1) remaining queries - in 
practice he may require slightly less as when he gets some levels with only 1 
element he doesn’t need to continue to query them. 


Hence in total this algorithm required at most the following number of queries 
n -I- 2dn' = n + 2n' log^,/ (n') = O{v! log n'). 

1.2. Multiple output instruments 

We now turn our attention to scales that return multiple elements. These ac¬ 
cept as input k elements from X and return a set of size s containing the 
t^'^... elements. We refer to such a scale as a {k,ti,..., tg) scale. 

In this case we shall outline an algorithm that works in similar stages to the sin¬ 
gleton output case, although in each case more work shall be needed to achieve 
the same ends. We shall again make use of the initial and final segments, which 
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with slight abuse of notation we again denote by S and L. This time however S 
consists of the first — 1 elements, and L of the final k — tg elements. Together 
they therefore comprise a set of fc — 1 — {tg — ti) elements. It is possible if n 
is small compared to k that there can be other elements that the user cannot 
distinguish the order of. If, for example, we consider a (7, 2, 6 ) instrument on 8 
elements then none of {xi, X 4 , ccs, xs} will ever included in an output, so as well 
as the initial and final segments we have a middle segment that is indistinguish¬ 
able. However in our case we think of n as being arbitrarily large compared to 
k; indeed we consider the asymptotics as n —>■ 00 , and if n > 2 A: then no such 
middle set of indistinguishable elements can exist. 

Stage 2.1. Determine the elements that comprise S U L. 

Method: Again, as in the singleton case. S' U T comprise the elements that are 
never in the output of any query. So the user can begin by repeatedly querying 
uneliminated elements, and discarding the elements contained in the outputs. 
In the singleton case, this worked until there were k — 1 elements left, and these 
comprise exactly S U L. In this case however the user can only eliminate ele¬ 
ments until he has k — s left, and in general SU L have k — l — {tg — ti) elements. 
These are the same li tg — ti = s — 1, which occurs if the numbers H, t 2 ■ ■ ■ ,ts 
are consecutive. In that case Stage 2.1 terminates at this point. 

li ti,... ,tg are not consecutive then the user has a little more work to do. After 
carrying out the above he has k — s candidates left for S U L, which contains 
S' U L and some extra elements which he wishes to eliminate. To do so we use 
an inductive approach. Specifically we shall show that if we have a set of k — a 
candidates with a < s, {ci,..., Ck-a}, that contains at least 1 element not in 
S U L then we can eliminate 1 more. This clearly suffices to eliminate all those 
not in S U L. Note that if we query {ci} along with a set of size a, as we get 
at least a outputs they must either consist exactly of the a additional elements, 
or include and thus eliminate an additional candidate. So to achieve the induc¬ 
tive aim, pick any set of ( 2 a — 1 ) already eliminated elements, which we denote 
{ei,..., e 2 o-i}, and carry out all the queries involving a of the additional 

elements along with the candidates. Let x be an element in {cj} not in S' U L, 
and note that by the pigeonhole principle either at least a of the {ei} are lower 
than X in the ordering, or at least a of them are greater - without loss of gener¬ 
ality we assume the former, and relabelling if necessary we assume that the set 
{ei,..., Ca} is among them. Hence when we query {ci,..., Ck-a, ei,..., eo}, all 
of L and x are greater than all of the additional elements, hence the element 
cannot be one of the additional elements and so must lie in the {ci}, eliminating 
it. This process can continue until the remaining candidates are exactly S U L, 
after this point there does not exist any x G {ci,..., Ck-a} \ (SU L). Hence this 
process ends exactly with S' U L being identified. 

Stage 2.2. Partition SUL into S and L, and if possible identify which is which. 
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Method: We take exactly the same approach as in the singleton case. Pick any 
set of size k — 1 taken from X\{S L), which we denote oi,..., Ok-i- The 
user carries out the [S' U L| queries consisting of this reference set taken with 
one element from S' U L. If the element from S U L was in S then this returns 
..., while if it was in L it returns {at^,... Although the 

user cannot immediately distinguish these they are clearly distinct, so he can 
partition S U L into S and L according to the multiplicites of the responses. 

In the singleton case the user could also determine which of these sets was S 
and which was L in the assymmetric case, since that implied they would be of 
different sizes. This is more complicated in the multiple-output situation, since 
you could have ti +ts = k + 1, giving [S'! = \L\, but still have an assymetric in¬ 
strument as a result of some other elements in the output. In practice it doesn’t 
matter which is which, and so for the moment we do not address the question 
of how to distinguish them in this algorithm. We will however address this at 
the end of this section for completeness. 

Stage 2.3. Use S to determine the order of the remaining elements. 

Method: If S and L are of different sizes, then the following works: Let S' be 
the set of the first — 1 elements of X, noting that this includes S. If the user 
can identify S' then he can reduce our scale to a (fc', 1) scale by insisting on 
always including S' in any query - this fills up the first ts — 1 slots of the scale, 
and means that it will always return some subset of S' (easily ignored) along 
with the smallest element of the remainder. Defining AT' as X \ S' he can then 
use this (fc', 1) scale to sort X' \ L in 0(nlogn) steps as per Stage 1.3 of the 
singleton output algorithm. This sorts the majority of X (assuming as always 
that n is large compared to k. To sort the remaining elements in S" \ S' is then 
straightforward since the user will have identihed the final k elements of AT, so 
can create an (k", k") instrument by including k — ti elements of X. This can 
be used to sort the remaining elements. 

It just remains therefore to say how to find S'. If U — 1 divides tg — 1 then S' 
is easy to find by repeatedly removing the smallest elements from X. In Stages 
2.1 and 2.2 we outlined how to identify S i.e. the smallest ti — 1 elements in 
X. By removing these and repeating the process the user can identify the next 
ti — 1 elements repeatedly until he has found the first U — 1 elements. If U — 1 
does not divide tg — 1 then the same process can be applied, except that when 
the user needs fewer than U — 1 more elements at the end to top off S', he leaves 
in some of the previous — 1 set that he removed so that he requires something 
of the correct size. 

If S and L are of the same size then at the end of Stage 2.2 the user has parti¬ 
tioned S' U L into sets A and B, which are S and L but he doesn’t know which 
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is which. We suggest the following: he makes an arbitrary assignment, claiming 
that A is S. Discarding this and repeating Stages 2.1 and 2.2 on the remaining 
X \ A elements will return two more sets, A' and B. Note that this will be the 
same B as before, so he can identify A' and discard it again. Continuing in this 
manner he will eventually find a set that is either S' or L', as above. He can 
assume this is S', and use it to form what he thinks is an (fc',1) instrument. 
Using this he can sort the remaining elements as above, and come up with an 
ordering for X. This ordering will either be correct, or exactly the reverse of 
the correct ordering if his initial theory that A was S was wrong. He can check 
which of these is true if he had an asymetric instrument by carrying out any 
query of k elements taken from X \{S A L), and seeing if the answer agrees 
with his opinion on what the ordering is. If it does not, he simply reverses the 
ordering. 


It now remains to analyse how long this shall take. Stage 2.1 takes 


n— {k—s) 
s 


queries initially to get down to fc — s elements. To get from there to S' U L 
involves removing at most k elements, and removing each takes at most 
queries, so in total this takes at most (2fc)^+^ queries. Stage 2.2 then takes 
|SU L| = k — 1 — {ts — ti) queries. Hence between them Stages 2.1 and 2.2 take 
in total a linear number of queries in n with an additional number of queries 
that is solely a function of k. As we consider the situation where n is large and 
k is small and fixed, this is effectively a linear number of queries in n. 


Stage 2.3 takes as most k runs of Stages 2.1 and 2.2 to identify S', which is 
therefore still linear in n. Having identified S' it then takes O(nlogn) steps to 
sort the set X'. The hnal sorting of the remanent, and the extra query in the 
case where S and L are the same size, clearly only take a number of steps that 
is a function of k, hence overall this algorithm also runs in time O(nlogn) as 
required. 


As promised, we now consider how to distinguish S from L in the case of an 
assymetric instrument in Stage 2.2. Let p be the smallest index that makes the 
instrument non-symmetric, in the sense that only one of the and (fc+1 —p)**' 
elements are in the output. Without loss of generality we shall assume it is the 
case that the p*^ is in the output and its reflection (inside the scale) is not, to 
ease notation. We define Si = S, Li = L and Xi = X \ {S U L), similarly 
to before, and then recursively define Si, Li and Xi to be the initial, hnal and 
middle segments of Xi-i respectively. Hence for any index z A is thus composed 
of Uj=i Uj=i Note that given that Stages 2.1 and 2.2 determined 

{S', £} from X, they can be repeated to determine {Si, Li) and Xi from Xi-i. 
Hence the user can build up as many pairs of sets as he wants, all of which 
form the initial and hnal segments of the remainder of the full set, provided n 
is sufficiently large. 


We use this idea to show how to hnd S and L. First the user determines 



{Si, Li} for all 1 ^i^p + k — 2. He can then identify Sp by querying a set 
containing one element from each of the sets in the pairs {5'i,Li}, {S 2 ,L 2 }, 

..{Sp-i, Lp-i}, a single element from one of the sets in {Sp,Lp} and then 
balancing elements from Xp to fill out the instrument. If the element he picked 
from {Sp, Lp} was from Sp then it will be in the output, if it was from Lp 
then it won’t, enabling him to identify Sp and Lp. He can then repeat this to 
{S'p+i,Lp+i}, {Sp+ 2 ,Lp+i}, ..., {S'p+fe,Lp+fc}, and thus identify exactly the sets 
Sp,Sp+i,, Sp+k- 2 - Now identifying S is simple, since he just needs to take 
one element from one of {S, L}, and query it along with k — 1 elements, taken 
one each from Sp, S'p+i, •.., Sp+k- 2 - The result of this query will determine if 
the element from {5, L} was from S or L merely by looking at which of the other 
elements was returned, and hence he can identify which set is S and which is 
L. As p is at most k/2, if he wished to do this it would require at most 3fc/2 
additional runs of Stages 2.1 and 2.2, which is still therefore linear in n and so 
would not materially affect the running time of the algorithm for large n. 

2. Offline Algorithms 

We now turn our attention to offline algorithms. In this situation the user must 
specify the full list of queries that he wishes to carry out in advance, and then 
receives all the answers simultaneously afterwards. As we saw in Section 1, if 
the user knows the results of all the possible queries then he can determine es¬ 
sentially the full ordering (excluding S and L as before), since he can follow any 
of the online algorithms, looking up the results of any query from his bank of 
known query results. We concern ourselves with trying to minimise the number 
of queries that he must request in order to determine this ordering. 

We begin by considering a singleton-scale, which returns a singleton output. 

2.1. Singleton Output Scales 

In this case, as in the online section on singleton output scales, the user is given 
a (fc, t) scale that accepts a fc-set as input and returns the smallest element. 
Again for simplicity we assume that t ^ k/2, if not then similar analysis fol¬ 
lows, occasionally replacing the word ‘smallest’ with ‘largest’ and, where we 
have comments about filling the scale from the lower elements, instead filling it 
from the higher. 

We first note a simple lower bound, namely that if there is any t-set that is not 
included in a query then there are some orderings that the algorithm would not 
be able to distinguish. This is because if the missed t-set comprised the first 
t elements of the ordering then no element of it would ever be included in the 
output of any set. Thus the user would not be able to tell which t — I subset 
of it formed S, and which element was the lowest element of A \ S'. The same 
applies if there was a. k — {t— I)-tuple that was not included in any query, since 
it could form the largest k — {t — \) elements, and then the user would not be 
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able to tell which was the largest element of X \ L. As t ^ fc/2 by assumption, 
this second set is larger, and so there are more possible k — {t — 1) tuples than 
there are t-tuples. Hence this forms the restriction that we appeal to. Noting 
that there are (fc — (t — l))-sets in any query, and as there are 

k—{t — l)-sets in total, this means that all algorithms must contain at least the 
following number of queries 


( ” 1 

We shall show that, in fact, there are algorithms that use this order of number of 
queries. We offer two for consideration, one that relies on recursively deducing 
the results of all possible queries, and thus the ordering, the second of which is 
direct and relies on determining the adjacencies of the ordering. We begin with 
the recursive algorithm. 

2.1.1. Recursive Algorithm 

Our algorithm works by fixing some set of r-elements, Y := {yi,... ,yr}, and 
requesting all the queries that involve Y and a {k — r) set from A \ F. We shall 
show that, provided r is not too large, then from this the user can deduce the 
result of an arbitrary query containing any (r — 1)-subset of F. If this holds 
then, by induction, the user can deduce the result of any query, and hence the 
full ordering. We prove this inductive claim by induction on r, beginning with 
the case r = 1. 

Theorem 1. If j/ is a fixed element and the results of all queries including y 
are known, then the result of a query on any set {oi,..., Ofc} can be deduced. 

Proof. Note that the claim is trivial if j/ G {oi,..., a^}, as this would mean that 
this exact query had taken place. So let us assume that y ^ {oi,..., a^}. We 
wish to deduce the value of at from queries of the form {y, oi, ..., Uk} \ {oi} 
for I < t < fc. We shall split into 4 cases for y and 3 cases for at in relation to 
at, and count how often we get various responses. These are summarised in the 
following grid: 



Multiplicity 

y < at-i 

y € {at- 

Resp 
1) Ot) 

onse 

y e {at,at+i) 

y > at +1 

at < at 

t-1 

at 

at 

y 

at +1 

at = at 

1 

Ot-l 

y 

y 

at+i 

at > at 

k — t 

at-i 

y 

at 

at 


Now, when performing these queries we get all the results from some column. 
So if, for example y < Ot-i, then we get at (t — I) times and Ot-i (1 + {k — t)) 
times. We can establish which column we are in, and thus how y compares 
with at, by looking at the multiplicities of the answers - if we have two different 
answers with multiplicities {t — 1) and (k — t + 1) then we are in one of the first 
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two columns, and if we get multiplicities t and (k — t) then we are in one of 
the last two columns. Further if we get the answer y for some of our queries 
we are in the middle two columns, if not we are in the outside columns. Thus 
we can determine which column we are in. Now by taking the result with the 
appropriate multiplicity ((t — 1) in the first two columns, and (fc — t) in the 
latter two) we can tell the value of at-, as required. We can summarise these in 
the following associated table: 


Case 

Multiplicities 

Mult, of at 

y < at 

{t — l,k — t + 1) 

t - 1 

y > at 

(t, k — t) 

k — t 


The general case is somewhat tricky to see, as the case analysis gets very de¬ 
tailed. Instead, we present the case for r = 2, which covers most of the concepts 
that we appeal to, and then explain how the argument changes for a general r. 
The first key point is that just taking the queries involving x and y would not 
by itself be enough, as the user will also need to know which is larger out of x 
and y. But the following lemma gives a simple way to do that 

Lemma 2. Assume that we have a asymmetric scale. Let Zi, ..., Zk+i be (/e-I-1) 
fixed elements of our set. By querying all the subsets of them we can find 

two of them x and y such that neither x nor y are in 5" U L and we know x < y. 

Proof. Relabelling these reference elements according to the ordering, we note 
that any query of a subset of them will either return zt or zt+i- As these are 
possible responses to queries, neither can be a member of 5" or L so we take 
these as our x and y. It remains to show that the user can identify which is 
the smaller, but it is clear that the user will receive the answer zt {k + 1 — t) 
times, and Zt+i t times, enabling him to distinguish them if t But 

this must hold, otherwise the scale would be symmetric, which contradicts our 
assumption. □ 

We now need the next requirement, that given x and y, two fixed elements of 
the set such that the user knows that x < y, and all the queries involving this 
pair, the user can determine the results of any possible query, and hence as 
much of the ordering as could ever be possible. 

Theorem 3. If a; and y are two fixed elements such that x < y and the results 
of all queries including x and y are known then the result of a query on any set 
{ai,..., Ofe} can be deduced. 

Proof. Note that by Theorem 1 it suffices to show that we can find the result 
of any query involving x and an arbitrary set of other elements, {oi,..., Uk-i}. 
We shall proceed on this basis. Relabelling them we can refer to such a set as 
{bi,... ,bk}, noting that x is now one of the {&}, say bi. We wish to establish 
how to find which is bt from a set of queries in which we replace each of the b 
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apart from bi by y. The results for these queries - depending on whether the 
bj that y replaces is smaller than, equal to or larger than bt - are summarised 
in the following table - note that now the multiplicities vary according to the 
position of a: = 6^: 




IN 

X <bt 

dultiplici 
X = bt 

ties 

X > bt 

y < bt-i 

Resp 

y e {bt-iM) 

onses 

y € {bt,bt+i) 

y > bt+i 

b, 

<bt 

t - 2 

t - 1 

t-1 

bt 

bt 

y 

bt+i 

b, 

= bt 

1 

0 

1 

bt-i 

y 

y 

bt+1 

b, 

>bt 

k — t 

k — t 

k — t — 1 

ht-i 

y 

bt 

bt 


We can again combine multiplicities according to y < bt and y > bt- At first 
this looks like it won’t let us differentiate options, as we get some situations 
with the same multiplicities: 


Case 

Multiplicities 

Mult, of at 


X 

<bt 

(t-2,k-t + l) 

t-2 

y <bt 

X 

= bt 

1 

1 

t-l 


X 

>bt 

{t — l,k — t) 

t-1 


X 

<bt 

{t — l,k — t) 

k — t 

y>bt 

X 

= bt 

1 

1 

k — t 


X 

>bt 

{t,k — t — 1) 

k — t — 1 


However we can see that this doesn’t matter. Firstly the 2"*^ and 3“''^ rows of the 
above table are impossible, as our initial assumption was that x < y, so we can 
remove those. Secondly, we note that of the remaining 4 situations although 2 
have the same multiplicities, in either of those two cases we just take the solution 
with multiplicity k — t and conclude that this is bt. Hence we can identify bt for 
any arbitrary set of elements {6i,..., 6^} containing x, and hence by Theorem 
1 we can determine the order of the full set. □ 

We now give our method for the case r = 2. The user first picks a set of size 
fc + 1, and requests all the queries involving k of those - note that there are 
(fc + 1) of these. By Lemma 2 this will find him a pair x and y from this set 
such that he knows x < y. He also considers all possible 2-tuples from this k+1 
set, and for each pair requests all queries involving that pair. This means that, 
in particular, even though he is carries out these queries offline he will know 
the results of all possible queries involving x and y. In total this requires an 
additional (A: -I-1) -b (^Z^) queries, which is of the same order as (^Z^) > 
hence But by Theorem 3 from these he can deduce the result of any 

query involving just one of x and y, and hence by Theorem 1 he can determine 
as much of the ordering as he could have hoped. 

As advertised, we shall not give the full explicit argument for general r, as the 
analysis is tedious and not much more enlightening than the r = 2 case. We 
shall instead explain how to modify the r = 2 case. The first part is simple 
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enough - carrying out all the possible probes on a fc + 1 set guaranteed us a pair 
of elements x, y such that we knew their internal ordering. In general, carrying 
out all the probes on some (k + r—l) set guarantees us a set of r elements which 
we can completely order from these probes - this is simply seen by just taking 
those as our whole universe and using any argument such as that outlined in 
the online cases. 

The notation for the other part gets more involved. We require the following 
statement by induction. Let {xi, ... ,Xr} be our reference set which we know 
the complete ordering of. We want to be able to say that we can deduce the 
value of some query involving the first (r — 1) of the reference set and some set 
of elements {oi,..., ak-r+i} by considering all the queries containing the full r 
elements of the reference set and some {k — r) subset of the a^’s. Let us relabel 
the set {xi, ..., oi,..., Uk-r+i} as {6i,..., 6fc}, so that some of the b are 
taken from a;-elements and some from a-elements. We shall then consider all 
the queries in which we replace one of the a-elements by Xr- Note that when 
counting multiplicities we must consider where bt lies relative to our reference 
set - i.e. how many of them are below it, and whether or not one of them is 
bt- This gives rise to the following table of multiplicities, we have omitted the 
left-hand four columns as they are again the same as the above. 



# 

(r - 1) < bt 

of reference se 
(r - 1) < bt 

t smaller than 
(r - 2) < bt 

bt 

(r - 2) < bt 


bj < bt 

t — r 

t — r + 1 

t — r + 1 

t — r + 2 



1 

0 

1 

0 


bj > bt 

k — t 

k — t 

k — t — 1 

k — t — 1 




m 

^ of reference set smaller than bt 


m 

2<bt 

1 <bt 

l = bt 

0<bt 



t - 2 

t-2 

t - 1 

t - 1 



0 

1 

0 

1 

bj > bt 


k — t — r + i 

k — t — r + 2 

k — t — r + 2 

k — t — r + 1 


This then gives rise to the following table of multiplicities, where the lefthand 
column again corresponds to the number of reference elements below or equal 
to bt. 
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Case 

Multiplicities 

Mult, of at 

Xr < bt 

(r - 1) < bt 

{t — r, k — t + 1) 

t — r 

(r - 1) < bt 

{t — r + l,k — t) 

t — r + \ 

(r - 2) < bt 

{t — r + 1, k — t) 

t — r + 1 




l<bt 

(t — 2, fc — t — r + 3) 

t - 1 

l = bt 

+ 

1 

1 

1 

t - 1 

0<bt 

{t — l,k — t — r + 2) 

t - 1 

Xr > bt 

(r - 1) < bt 

{t — r + l,k — t) 

k — t 

(r - 1) < bt 

{t — r + 1, k — t) 

k — t 

(r - 2) < bt 

{t — r + 2, k — t — 1) 

k — t — 1 




1 <bt 

{t — 1, k — t — r + 2) 

k — t — r + 2 

l = bt 

{t — 1, k — t — r + 2) 

k — t — r + 2 

0<bt 

{t,k — t — r + 1) 

k — t — r + 1 


Again we can eliminate a large number of these situations. As we took to be 
the maximum of the fixed reference elements, in the first half of this table all but 
the top row disappear, since if Xr is the largest and Xr < bt then all the other 
(r — 1) of them must also be less than bt- In the second half each multiplicity is 
repeated twice, but we note that this doesn’t affect our analysis as this is coming 
from separately counting the case where bt is one of our reference elements and 
where it isn’t. This however doesn’t matter, as all we are interested in is the 
value of bt, and either way we take the answer with the larger multiplicity (i.e. 
the second, as t < |), and so recover bt- 

This works all the way down to r = t — 1. However when r = t this no longer 
works, as you could be unlucky and pick as your fixed elements S together with 
the smallest element left in the set. Then every probe would just give Xr as it 
will be the t**' element of any query. This is equivalent to noting in the above 
analysis that if all the first (r — 1) are less than bt then if Xr < bt you’ll end 
up trying to pick the element with multiplicity 0, which doesn’t exist. Hence in 
this case you won’t be able to identify bt- 

This matches the lower bound that we expect, so we conclude that this gives an 
construction requiring ' (fc-t+i) steps to sort the set, 

to find a fixed reference set of size (t — 1) that we can fully order and then 
Ck-t+i) further steps to carry out all possible queries with each possible set of 
{t — 1) fixed elements. This is as required. 

2.1.2. Adjacency algorithm 

The key concept behind the second algorithm that we present is that of knowing 
which elements are next to which in the ordering. We begin with the following 
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observation, which states that this would be sufficient to determine the full 
ordering of the element set. 

Observation 4. Let X := {cci,..., x„} be a set of ordered elements, with the 
ordering being fixed but unknown to a user. Assume that the user knows which 
elements are adjacent to which others, i.e. he is given a map ^ : A —5> 
such that 


{X 2 } i = 1 

{xn-i} i = n 

Then the user can deduce the ordering of the element set, up to reflection. 

Proof. Note that only 2 elements only have 1 neighbour, namely xi and x„. 
Hence the user can identify this pair easily. He picks one, and assumes it is 
xi- X 2 is then immediately given as the sole neighbour of xi. He then proceeds 
iteratively - assume he has identified xi,... ,Xr up to some number 1 ^ r < n, 
Xr+i is then the element in 4>{xr) which is not Xr-i, extending the ordering. 
He can repeat this until he finally finds x„, and ends up either with the correct 
ordering or, if his initial choice of xi was incorrect, the reflection of it. □ 

Hence it suffices to find a full list of the adjacencies to determine the ordering 
up to reflection. Note that, having done so if the instrument is asymmetric then 
any single query’s result will determine which of the two possible orderings the 
user has, and if the instrument was symmetric then this would be the best he 
could have hoped for anyway. As ever, a full list will not be possible, but he 
can try to find all the adjacencies within X \ (S U L). To do this the user will 
eliminate possible adjacencies for each element until only the actual adjacencies 
remain - and then appeal to the above observation to determine the ordering. 

We suggest the following approach. Consider two elements from X \ (S U L), x 
and y where x and y are adjacent. If any query returns the response x and then 
the same query is attempted with x replaced by y, then the second query must 
return the element y. Alternatively consider the situation where we have three 
elements, a, b and c, all taken from X \ (S U L), with a < b < c, and as of yet 
the user does not know anything about their adjacencies. Consider a query of 
the form {xi, ..., Xk- 2 } U {a, b} which returns the output a. This means that a 
query of {xi,..., Xk- 2 } U {c, 6}, with a replaced by c, cannot return a response 
of c since it would pick up b first. Hence if he carries out these two queries he 
will know that a cannot be adjacent to c - if it was by the first comment c would 
have had to be the response of the second query. However the existence of b 
between a and c ensures that the second query will not return c. 


^Where we use the standard notation to be the set of all sets containing 2 elements 

from X. 
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This motivates our approach - the user seeks a set of queries such that, for 
any triple (a, b, c) such that all three lie in X \ (S' U L) with a < 6 < c he can 
find some query containing a and b that returns a, and the same query with 
c replacing a. It suffices to fix some reference set of size t — 1, which we call 
j/i,..., Ut-i, and take all queries that contain these elements. If a, b, and c are 
all from X\{SU L), with a < b < c, then as at most t — I of the fixed elements 
{Ui} are less than a, the query that consists of these reference elements, a, b, 
enough elements from S to ensure that a is the t**' smallest and the remaining 
elements from L will return a. 

Provided that a and c are not in this fixed reference set, as we include all pos¬ 
sible queries containing the yi,, yt-i, the user will also see the result of the 
same query with a replaced by c, and will then be able to conclude that a is not 
adjacent to c. He will be able to do this for all the remaining c in X \ (S' U L), 
and thus be able to eliminate all of o’s non-neighbours. He will be left with a’s 
neighbours, and thus be able to deduce them. 

If a or c are in the reference set, then this will not work. However we can cir¬ 
cumvent this by carrying out three sets of queries, with disjoint fixed reference 
sets each time. Then, for any non-adjacent pair a and c, one of the three sets 
of queries must have neither a nor c in its reference set. Hence in at least one 
set of queries the fact that a and c are not adjacent will be revealed. Since the 
user will discover this for all the elements in \ (S' U L) which are not adjacent 
to a, he will be left with those that are, and thus will be able to determine the 
ordering using Observation 4. 

We note that this takes which is again queries, but 

with a much improved constant factor over the previous recursive structure. It 
is also computationally less complicated, taking approximately calculations 
to eliminate all the non-adjacencies, and then a linear number of steps to rebuild 
the ordering, while the previous method required potentially reconstructing all 
the (^) queries, a considerably larger task. 

2.2. Multiple Output Scales 

The question now arises of which of these algorithms also works in the Multiple- 
Output case, where the user is given a {k,ti,... ,ts) scale as in the online ana¬ 
logue, and asked to determine the ordering. The authors note that the recursive 
algorithm was computationally and conceptually complicated even in the sin¬ 
gleton output case - although we suspect it is possible to also use it for multiple 
output scales, the case analysis would make such an approach exceptionally te¬ 
dious. However the adjacency based algorithm works almost immediately with 
almost no modifications. Since it just relies on showing that certain things 
would have to be included in the output if certain adjacencies existed, the same 
remains true even if the scale returns more elements. The only change required 
is that the fixed reference set is of a different size - before it contained at most 
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t members, now it must contain at most the maximum of ~ 1 and k — ti 
elements. This just ensures that the fixed reference elements don’t take up so 
much of the scale that it’s possible for every query containing them to only give 
an output consisting of members of the reference set. That established, the 
same analysis as before works, and so such an instrument can determine the 
ordering in at most the following number of queries 


3 max 


[k-{ts-l)J’\ h J 
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